Sound-action symbolism in relation to precision manipulation and whole-hand grasp usage.

It has been suggested that actions can provide a fruitful conceptual context for sound symbolism phenomena, and that tight interaction between manual and articulatory processes might cause that hand actions, in particular, are sound-symbolically associated with specific speech sounds. Experiment 1 investigated whether novel words, built from speech sounds that have been previously linked to precision or power grasp responses, are implicitly associated with perceived actions that present precision manipulation or whole-hand grasp tool-use or the corresponding utilisation pantomimes. In the two-alternative forced-choice task, the participants were more likely to match novel words to tool-use actions and corresponding pantomimes that were sound-symbolically congruent with the words. Experiment 2 showed that the same or even larger sound-action symbolism effect can be observed when the pantomimes present unfamiliar utilisation actions. Based on this we propose that the sound-action symbolism might originate from the same sensorimotor mechanisms that process the meaning of iconic gestural signs. The study presents a novel sound-action phenomenon and supports the view that hand-mouth interaction might manifest itself by associating specific speech sounds with grasp-related utilisations.

Front Is High and Back Is Low: Sound-Space Iconicity in Finnish.

Previous investigations have shown various interactions between spatial concepts and speech sounds. For instance, the front-high vowel [i] is associated with the concept of forward, and the back-high vowel [o] is associated with the concept of backward. Three experiments investigated whether the concepts of forward/front and backward/back are associated with high- and low-pitched vocalizations, respectively, in Finnish. In Experiments 1 and 2, the participants associated the high-pitched vocalization with the forward-directed movement and the low-pitched vocalizations with the backward-directed movement. In Experiment 3, the same effect was observed in relation to the concepts of front of and back of. We propose that these observations present a novel sound-space symbolism phenomenon in which spatial concepts of forward/front and backward/back are iconically associated with high- and low-pitched speech sounds. This observation is discussed in relation to the grounding of semantic knowledge of these spatial concepts in the movements of articulators such as relative front/back-directed movements of the tongue.

Interaction between grasping and articulation: How vowel and consonant pronunciation influences precision and power grip responses.

Grasping and mouth movements have been proposed to be integrated anatomically, functionally and evolutionarily. In line with this, we have shown that there is a systematic interaction between particular speech units and grip performance. For example, when the task requires pronouncing a speech unit simultaneously with grasp response, the speech units [i] and [t] are associated with relatively rapid and accurate precision grip responses, while [ɑ] and [k] are associated with power grip responses. This study is aimed at complementing the picture about which vowels and consonants are associated with these grasp types. The study validated our view that the high-front vowels and the alveolar consonants are associated with precision grip responses, while low and high-back vowels as well as velar consonants or those whose articulation involves the lowering of the tongue body are associated with power grip responses. This paper also proposes that one reason why small/large concepts are associated with specific speech sounds in the sound-magnitude symbolism is because articulation of these sounds is programmed within the overlapping mechanisms of precision or power grasping.

Sound-Action Symbolism.

Recent evidence has shown linkages between actions and segmental elements of speech. For instance, close-front vowels are sound symbolically associated with the precision grip, and front vowels are associated with forward-directed limb movements. The current review article presents a variety of such sound-action effects and proposes that they compose a category of sound symbolism that is based on grounding a conceptual knowledge of a referent in articulatory and manual action representations. In addition, the article proposes that even some widely known sound symbolism phenomena such as the sound-magnitude symbolism can be partially based on similar sensorimotor grounding. It is also discussed that meaning of suprasegmental speech elements in many instances is similarly grounded in body actions. Sound symbolism, prosody, and body gestures might originate from the same embodied mechanisms that enable a vivid and iconic expression of a meaning of a referent to the recipient.

Annoyance, perception, and physiological effects of wind turbine infrasound.

Even though some individuals subjectively associate various symptoms with infrasound, there are very few systematic studies on the contribution of infrasound to the perception, annoyance, and physiological reactions elicited by wind turbine sound. In this study, sound samples were selected among long-term measurement data from wind power plant and residential areas, both indoors and outdoors, and used in laboratory experiments. In the experiments, the detectability and annoyance of both inaudible and audible characteristics of wind turbine noise were determined, as well as autonomic nervous system responses: heart rate, heart rate variability, and skin conductance response. The participants were divided into two groups based on whether they reported experiencing wind turbine infrasound related symptoms or not. The participants did not detect infrasonic contents of wind turbine noise. The presence of infrasound had no influence on the reported annoyance nor the measured autonomic nervous system responses. No differences were observed between the two groups. These findings suggest that the levels of infrasound in the current study did not affect perception and annoyance or autonomic nervous system responses, even though the experimental conditions corresponded acoustically to real wind power plant areas.

Automatic inhibition of habitual response associated with a non-target object while performing goal-directed actions.

This study is devoted to investigating mechanisms that inhibit habituated response associated with affordance of a non-target while executing action directed to a target. In four experiments, a paradigm was used that required a rapid left- or right-hand response according to the direction of the target arrow presented simultaneously or in close temporal proximity to a non-target whose handle position afforded grasping with the left or right hand. In general, responding was decelerated and more erroneous when the handle position was compatible with the responding hand. This effect of response inhibition was removed when the delay between the non-target offset and target onset was longer than 200 ms, and reversed into response facilitation when the target onset was delayed for 400-600 ms. The study suggests that processes that control withholding habitual response associated with affordance of a non-target utilise response inhibition mechanisms overlapping with those involved in behavioural control of the stop-signal task. This response inhibition is triggered automatically and directly by affordance of a non-target without preceding response excitation associated with this affordance cue.

The Sound of Grasp Affordances: Influence of Grasp-Related Size of Categorized Objects on Vocalization.

Previous research shows that simultaneously executed grasp and vocalization responses are faster when the precision grip is performed with the vowel [i] and the power grip is performed with the vowel [ɑ]. Research also shows that observing an object that is graspable with a precision or power grip can activate the grip congruent with the object. Given the connection between vowel articulation and grasping, this study explores whether grasp-related size of observed objects can influence not only grasp responses but also vowel pronunciation. The participants had to categorize small and large objects into natural and manufactured categories by pronouncing the vowel [i] or [ɑ]. As predicted, [i] was produced faster when the object's grasp-related size was congruent with the precision grip while [ɑ] was produced faster when the size was congruent with the power grip (Experiment 1). The effect was not, however, observed when the participants were presented with large objects that are not typically grasped by the power grip (Experiment 2). This study demonstrates that vowel production is systematically influenced by grasp-related size of a viewed object, supporting the account that sensory-motor processes related to grasp planning and representing grasp-related properties of viewed objects interact with articulation processes. The paper discusses these findings in the context of size-sound symbolism, suggesting that mechanisms that transform size-grasp affordances into corresponding grasp- and articulation-related motor programs might provide a neural basis for size-sound phenomena that links small objects with closed-front vowels and large objects with open-back vowels.

Connecting directional limb movements to vowel fronting and backing.

It has been shown recently that when participants are required to pronounce a vowel at the same time with the hand movement, the vocal and manual responses are facilitated when a front vowel is produced with forward-directed hand movements and a back vowel is produced with backward-directed hand movements. This finding suggests a coupling between spatial programing of articulatory tongue movements and hand movements. The present study revealed that the same effect can be also observed in relation to directional leg movements. The study suggests that the effect operates within the common directional processes of movement planning including at least tongue, hand and leg movements, and these processes might contribute sound-to-meaning mappings to the semantic concepts of 'forward' and 'backward'.

Connection between movements of mouth and hand: Perspectives on development and evolution of speech.

Mounting evidence shows interaction between manipulative hand movements and movements of tongue, lips and mouth in a vocal and non-vocal context. The current article reviews this evidence and discusses its contribution to perspectives of development and evolution of speech. In particular, the article aims to present novel insight on how processes controlling the two primary grasp components of manipulative hand movements, the precision and power grip, might be systematically connected to motor processes involved in producing certain articulatory gestures. This view assumes that due to these motor overlaps between grasping and articulation, development of these grip types in infancy can facilitate development of specific articulatory gestures. In addition, the hand-mouth connections might have even boosted the evolution of some articulatory gestures. This account also proposes that some semantic sound-symbolic pairings between a speech sound and a referent concept might be partially based on these hand-mouth interactions.

Reaching and grasping with the tongue: Shared motor planning between hand actions and articulatory gestures.

Research has shown connections between articulatory mouth actions and manual actions. This study investigates whether forward-backward hand movements could be associated with vowel production processes that programme tongue fronting/backing, lip rounding/spreading (Experiment 1), and/or consonant production processes that programme tongue tip and tongue dorsum actions (Experiment 2). The participants had to perform either forward or backward hand movement and simultaneously pronounce different vowels or consonants. The results revealed a response benefit, measured in vocal and manual reaction times, when the responses consisted of front vowels and forward hand movements. Conversely, back vowels were associated with backward hand movements. Articulation of rounded versus unrounded vowels or coronal versus dorsal consonants did not produce the effect. In contrast, when the manual responses of forward-backward hand movements were replaced by precision and power grip responses, the coronal consonants [t] and [r] were associated with the precision grip, whereas the dorsal consonant [k] was associated with the power grip. We propose that the movements of the tongue body, operating mainly for vowel production, share the directional action planning processes with the hand movements. Conversely, the tongue articulators related to tongue tip and dorsum movements, operating mainly for consonant production, share the action planning processes with the precision and power grip, respectively.

Upper nasal hemifield location and nonspatial auditory tones accelerate visual detection during dichoptic viewing.

Visual performance is asymmetric across the visual field, but locational biases that occur during dichoptic viewing are not well understood. In this study, we characterized horizontal, vertical and naso-temporal biases in visual target detection during dichoptic stimulation and explored whether the detection was facilitated by non-spatial auditory tones associated with the target's location. The detection time for single monocular targets that were suppressed from view with a 10 Hz dynamic noise mask presented to the other eye was measured at the 4° intercardinal location of each eye with the breaking Continuous Flash Suppression (b-CFS) technique. Each target was either combined with a sound (i.e., high or low pitch tone) that was congruent or incongruent with its vertical location (i.e., upper or lower visual field) or presented without a sound. The results indicated faster detection of targets in the upper rather than lower visual field and faster detection of targets in the nasal than temporal hemifield of each eye. Sounds generally accelerated target detection, but the tone pitch-elevation congruency did not further enhance performance. These findings suggest that visual detection during dichoptic viewing differs from standard viewing conditions with respect to location-related perceptual biases and crossmodal modulation of visual perception. These differences should be carefully considered in experimental designs employing dichoptic stimulation techniques and in display applications that utilize dichoptic viewing.

Inter-limb coupling of proximal and distal hand actions.

Past studies have revealed connections in directional programming between hands. The present study investigated whether there could also be interaction in programming proximal and distal components of a prehensile hand action. In Experiment 1, the participants performed simultaneously either a push or pull response with the left hand and the grip closure or opening with the right hand. In Experiment 2, the push and pull responses of the left hand were performed together with the precision or power grip responses of the right hand. The participants showed preference, measured in reaction times, to couple the push response with the grip opening and the precision grip, whereas the pull response was associated with the grip closure and the power grip. The study shows for the first time a systematic interaction in proximal and distal prehensile components between two hands. We propose that these effects reflect inter- and intra-limb connections between the representations that prepare the arm extension for the outward reaching, the finger extension for the grip opening, and the motor processes that prepare the precision grip. Conversely, there appear to be connections between the representations that prepare the arm flexion for the inward directed hand movements, the flexion of the thumb and the fingers for the grip closure, and flexion of four fingers for the power grip.

Mismatch negativity (MMN) to speech sounds is modulated systematically by manual grip execution.

Manual actions and speech are connected: for example, grip execution can influence simultaneous vocalizations and vice versa. Our previous studies show that the consonant [k] is associated with the power grip and the consonant [t] with the precision grip. Here we studied whether the interaction between speech sounds and grips could operate already at a pre-attentive stage of auditory processing, reflected by the mismatch-negativity (MMN) component of the event-related potential (ERP). Participants executed power and precision grips according to visual cues while listening to syllable sequences consisting of [ke] and [te] utterances. The grips modulated the MMN amplitudes to these syllables in a systematic manner so that when the deviant was [ke], the MMN response was larger with a precision grip than with a power grip. There was a converse trend when the deviant was [te]. These results suggest that manual gestures and speech can interact already at a pre-attentive processing level of auditory perception, and show, for the first time that manual actions can systematically modulate the MMN.

Connections of Grasping and Horizontal Hand Movements with Articulation in Czech Speakers.

We have recently shown in Finnish speakers that articulation of certain vowels and consonants has a systematic influence on simultaneous grasp actions as well as on forward and backward hand movements. Here we studied whether these effects generalize to another language, namely Czech. We reasoned that if the results generalized to another language environment, it would suggest that the effects arise through other processes than language-dependent semantic associations. Rather, the effects would be likely to arise through language-independent interactions between processes that plan articulatory gestures and hand movements. Participants were presented with visual stimuli specifying articulations to be uttered (e.g., A or I), and they were required to produce a manual response concurrently with the articulation. In Experiment 1 they responded with a precision or a power grip, whereas in Experiment 2 they responded with a forward or a backward hand movement. The grip congruency effect was fully replicated: the consonant [k] and the vowel [α] were associated with power grip responses, while the consonant [t] and the vowel [i] were associated with precision grip responses. The forward/backward congruency effect was replicated with vowels [α], [o], which were associated with backward movement and with [i], which was associated with forward movement, but not with consonants [k] and [t]. These findings suggest that the congruency effects mostly reflect interaction between processes that plan articulatory gestures and hand movements with an exception that the forward/backward congruency effect might only work with vowel articulation.

Preparation and execution of teeth clenching and foot muscle contraction influence on corticospinal hand-muscle excitability.

Contraction of a muscle modulates not only the corticospinal excitability (CSE) of the contracting muscle but also that of different muscles. We investigated to what extent the CSE of a hand muscle is modulated during preparation and execution of teeth clenching and ipsilateral foot dorsiflexion either separately or in combination. Hand-muscle CSE was estimated based on motor evoked potentials (MEPs) elicited by transcranial magnetic stimulation (TMS) and recorded from the first dorsal interosseous (FDI) muscle. We found higher excitability during both preparation and execution of all the motor tasks than during mere observation of a fixation cross. As expected, the excitability was greater during the execution phase than the preparation one. Furthermore, both execution and preparation of combined motor tasks led to higher excitability than individual tasks. These results extend our current understanding of the neural interactions underlying simultaneous contraction of muscles in different body parts.

Systematic Influence of Perceived Grasp Shape on Speech Production.

Previous research has shown that precision and power grip performance is consistently influenced by simultaneous articulation. For example, power grip responses are performed relatively fast with the open-back vowel [a], whereas precision grip responses are performed relatively fast with the close-front vowel [i]. In the present study, the participants were presented with a picture of a hand shaped to the precision or power grip. They were required to pronounce speech sounds according to the front/above perspective of the hand. The results showed that not only the grip performance is affected by simultaneously pronouncing the speech sound but also the production of speech sound can be affected by viewing an image of a grip. The precision grip stimulus triggered relatively rapid production of the front-close vowel [i]. In contrast, the effect related to the power grip stimulus was mostly linked to the vertical dimension of the pronounced vowel since this stimulus triggered relatively rapid production of the back-open vowel [a] and back-mid-open vowel [o] while production of the back-close vowel [u] was not influenced by it. The fact that production of the dorsal consonant [k] or coronal consonant [t] were not influenced by these stimuli suggests that the effect was not associated with a relative front-back tongue shape of the articulation in the absence of changes in any vertical articulatory components. These findings provide evidence for an intimate interaction between certain articulatory gestures and grip types, suggesting that an overlapping visuomotor network operates for planning articulatory gestures and grasp actions.

Interaction in planning vocalizations and grasping.

Previous studies have shown a congruency effect between manual grasping and syllable articulation. For instance, a power grip is associated with syllables whose articulation involves the tongue body and/or large mouth aperture ([kɑ]) whereas a precision grip is associated with articulations that involve the tongue tip and/or small mouth aperture ([ti]). Previously, this effect has been observed in manual reaction times. The primary aim of the current study was to investigate whether this congruency effect also takes place in vocal responses and to investigate involvement of action selection processes in the effect. The congruency effect was found in vocal and manual responses regardless of whether or not the syllable or grip was known a priori, suggesting that the effect operates with minimal or absent action selection processes. In addition, the effect was observed in vocal responses even when the grip was only prepared but not performed, suggesting that merely planning a grip response primes the corresponding articulatory response. These results support the view that articulation and grasping are processed in a partially overlapping network.

Excitability of hand motor areas during articulation of syllables.

It is known that articulating different syllables is linked to different grasp actions, e.g. [ti] is linked to precision grip, and [kɑ] to power grip. The aim of the present study was to test whether articulating or hearing these syllables would result in an increased activity in the representation of hand muscles involved in these two actions in a muscle-specific manner. To this end, we used transcranial magnetic stimulation (TMS) to investigate changes in the excitability of the left primary motor cortex (M1) innervating hand muscles while participants articulated or listened to meaningless syllables, listened to a metronome, or observed a fixation cross. The motor-evoked potentials of two hand muscles associated with either a precision or power grip exhibited significantly greater amplitudes during articulation than in passive listening, metronome, and fixation cross conditions. Moreover, these muscles exhibited similar patterns of excitability during articulation regardless of which syllable was articulated. The increased excitability of the left M1 hand area during articulation, but not during perception of the syllables, might be due to the cortico-cortical interaction between the motor representations of oral organs with the hand area.

Attention-shift vs. response-priming explanations for the spatial cueing effect in cross-modal tasks.

The task-irrelevant spatial location of a cue stimulus affects the processing of a subsequent target. This "Posner effect" has been explained by an exogenous attention shift to the spatial location of the cue, improving perceptual processing of the target. We studied whether the left/right location of task-irrelevant and uninformative tones produces cueing effects on the processing of visual targets. Tones were presented randomly from left or right. In the first condition, the subsequent visual target, requiring response either with the left or right hand, was presented peripherally to left or right. In the second condition, the target was a centrally presented left/right-pointing arrow, indicating the response hand. In the third condition, the tone and the central arrow were presented simultaneously. Data were recorded on compatible (the tone location and the response hand were the same) and incompatible trials. Reaction times were longer on incompatible than on compatible trials. The results of the second and third conditions are difficult to explain with the attention-shift model emphasizing improved perceptual processing in the cued location, as the central target did not require any location-based processing. Consequently, as an alternative explanation they suggest response priming in the hand corresponding to the spatial location of the tone. Simultaneous lateralized readiness potential (LRP) recordings were consistent with the behavioral data, the tone cues eliciting on incompatible trials a fast preparation for the incorrect response and on compatible trials preparation for the correct response.

Selective Influences of Precision and Power Grips on Speech Categorization.

Recent studies have shown that articulatory gestures are systematically associated with specific manual grip actions. Here we show that executing such actions can influence performance on a speech-categorization task. Participants watched and/or listened to speech stimuli while executing either a power or a precision grip. Grip performance influenced the syllable categorization by increasing the proportion of responses of the syllable congruent with the executed grip (power grip-[ke] and precision grip-[te]). Two follow-up experiments indicated that the effect was based on action-induced bias in selecting the syllable.